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ABSTRACT 

Data fusion and sensor management systems have largely been implemented with centralized 
and hierarchical architectures. Numerical and statistical methods are the most common data 
fusion methods found in these systems. Some recent research has emphasized distributed and 
decentralized systems, be it using analytical/quantitative techniques or qualitative reasoning 
methods. It appears that little has been done to define generic paradigms and theories to apply 
qualitative reasoning as an inference mechanism in distributed and decentralized systems. Based 
on other work by the authors, a sensor may be treated as a highly autonomous (decentralized) 
unit. In this context, this paper describes a distributed sensor-data-fusion paradigm and theory 
based on a previously developed theory to model sensors as highly autonomous units. Generic 
procedures are defined to reason and make decisions at the qualitative level. Therefore, fusion is 
done at a high-qualitative level, thus making it possible the implementation of intuitive (effective) 
methods to monitor, diagnose, and compensate processes/systems and their sensors. This 
paradigm facilitates distribution of intelligence (code and hardware) to the sensor level and peer- 
to-peer communication among sensors, controllers, and other devices in the system. 


INTRODUCTION 

Sensor fusion can be seen in two levels. One where only one physical sensor participates, 
and fusion entails interpretation of the current data point using a window of previously read 
points. That is, measurements from one sensor over a period of time (The measurement history 
of a sensor) are used to improve the quality and integrity of the current data point. A second level 
of sensor fusion implies participation of a group of physical sensors measuring the same or 
different parameters of a process. In this case, fusion entails the combination of the current data 
point of all sensors to improve the quality and integrity of one or many sensors of interest in the 
group. To further improve the measurement, both of these levels may be used simultaneously. 
For example, nuclear power plants have three of the same sensors in one location (three 
redundant sensors of the same kind at the same location and for the same measurement purpose). 
Three steps are taken to insure data integrity and accuracy. First data points from each sensor are 
averaged over a number of measurements. Second, a voting procedure eliminates the one average 
that is less similar. Third, the two remaining averages are averaged again to obtain the best 
interpretation of the measurement. 


There are many analytical and statistical methods to perform sensor fusion. These use 
estimating tools such as equations describing physical phenomena monitored by the sensors, 
curve fitting methods, Kalman filtering, averaging, etc. These methods have been further 
augmented with expert-system type qualitative tools to reason using prepositional logic about the 
operation of the sensors and the measurand. More recent techniques that combine both logic and 
analytical methods such as Fuzzy Logic (Huntsberger, Jayaramamurthy, 1987) and Qualitative 
Process Theory (QPT) (Forbus, 1982, 1986) provide a more generic (formalized theory) approach 
to use of logic reasoning in sensor fusion. 

Yet another issue in sensor fusion refers to the architecture used to implement a particular 
scheme. Sensor fusion architectures may be centralized-hierarchical or distributed-hierarchical. 
Architectures have tended to be centralized, since data from each sensor unit is usually 
transferred to a central place to be processed there along with data from other sensors. A 
distributed hierarchy means that each sensor unit processes its own data and sends a rich 
(integrated) set of information to any entity that may need to use it, including other sensors. In 
fact, each sensor in a distributed system has access to models of the processes in which the 
particular sensor is involved, along with information from all other sensors associated with the 
same processes. This approach defines each sensor as an autonomous agent in the context of 
particular processes. Moreover, it defines effectively a distributed-highly-decentralized system. 

This paper presents a sensor fusion method that is distributed-highly-decentralized (DHD). It 
defines a sensor-process-controller physical network where each sensor is a highly autonomous 
sensor (HAS). Further, each HAS in the network can extract qualitative behaviors associated 
with the measurand and itself and thus fusion is done at a high qualitative level using logic 
reasoning about sensor and measurand behaviors. Moreover, the HAS model is generic and 
allows easy instantiation of any type of physical sensor as HAS. This is an important difference 
with respect to other models that use expert systems, in which extensive knowledge bases for 
each sensor-measurand pair must be created. 


BACKGROUND 

We are interested in using sensor fusion to interpret signals more accurately, to monitor 
behaviors of the sensors and measurands, to diagnose existing malfunctions or abnormal 
behaviors, and to predict future behaviors. As mentioned in the introduction, all of the existing 
sensor fusion methods known to the authors tend to interpret and monitor a system or process 
either totally using numerical and statistical tools or only logic reasoning. Exceptions to this 
statement include Qualitative Process Theory developed by Kuipers and implemented as QSIM 
(for qualitative simulation) (Kuipers, 1986) and other methods based on what is denominated 
qualitative physics in the artificial intelligence community (de Kleer, Williams, 1986). However, 
these theories have been developed as generic tools to reason qualitatively about physical 
phenomena. We are using some ideas and concepts from these researchers and are applying them 
to develop a DHD fusion scheme for a network of sensors and associated processes or systems. 

Kuipers’ QPT was developed to describe qualitatively relationships among parameters of a 
process or system, which are normally described by analytical differential equations. 

Subsequently these qualitative descriptions may be used to reason and predict future behavior of 
the process/system. Later, Kuipers also developed MIMIC (Dvorak, 1989,1992)(Dvorak, 

Kuipers, 1996), in which he applied QPT specifically to the task of monitoring a process or 
system with multiple sensors, de Kleer's ATMS (Asumption Based Truth Maintenance System) 
(de Kleer, Williams, 1986), GDE (General Diagnostic Engine, Forbus' ATMI (1982) and 
SIMGEN(1987), DeCoste's DATMI (Dynamic Across Time Measurement and Interpretation) 
(1991), Durrant- Whyte’s distributed decision making team( 1990a, 1990b), Rodney Brooks’ 
Subsumption Architecture, are all seemingly diverse approaches aimed explicitly at addressing 
the fundamental problem of how to combine, in the best possible manner, diverse and uncertain 


sensor measurements and other information available in a multi-sensor system. The ultimate goal 
is to enable the system to estimate or make inferences concerning the state of the sensor’s 
physical nature. 

We present a fusion method that is inspired by QPT in its generality as a theory to reason 
about physical phenomena, but use typical system qualitative behaviors (responses to various 
inputs) instead of the qualitative equivalents of differential equations. The inspiration for this 
approach is DeCoste’s DATMI (1991)(Dynamic Across Time Measurement and Interpretation). 
It requires the use of sensor models that are able to extract qualitative behaviors of themselves 
and of the measurands from the raw data. These sensor models have been previously developed 
by the first author and other researchers and are denominated Highly Autonomous Sensors (HAS) 
(Figueroa, Mahajan, 1994a, 1994b, 1995, Figueroa, Yuan, 2000). In this context, this paper 
describes a distributed hierarchical sensor-data-fusion theory that employs qualitative methods to 
interpret measurements, to reason, and to make decisions based on the measurement 
interpretations. In this manner, fusion is done at a high-qualitative level, thus making it possible 
the implementation of intuitive (effective) methods to monitor, diagnose, and compensate 
processes/systems and their sensors. 

GENERAL FUNCTIONAL ARCHITECTURE 

The proposed architecture is shown in Fig. 1. It is hard to imagine a system without top 
management level even though the lowest HAS level should operate in a highly autonomous 
manner. This contrasts with Durrant- Whyte’s distributed sensor fusion (1988), which has no 
higher-level organization, and each sensor node also has to generate a control strategy. The task 
of the fully distributed sensor network as described in (Durrant- Whyte, 1988, Mutambara, 1998) 
is to control the system. The task for our system is to provide useful information for the operator 
or manager who is responsible for reacting promptly and correctly when abnormal behaviors 

occur, and to execute corrective 
action on the sensor operation 
when a fault has been evaluated 
and described with appropriate 
reliability and accuracy. 


Many engineering systems 
consist of distributed devices, 
and decentralized architectures 
present us with a way to build 
global systems of the devices. In 
a fully decentralized system, all 
information is processed locally. 
Our decentralized sensor data- 
fusion system consists of a 
network of sensor nodes (highly 
autonomous sensors), each with its own processing capabilities. Sensors transfer each other and 
to a higher-level system only needed information that is not available within their particular 
domain. The bus-based architecture shown is typical of modern networked systems. Each sensor 
may be associated with one or more processes. And each process may be associated with other 
processes through input/output parameters. Processes need not be physically separate, but 
interaction among them should occur through input/output parameters. 

As shown in Fig. 1 , at the lowest level, we have a distributed network of HAS’s which have 
knowledge of the process model in which they partake. Each HAS detects normal and abnormal 
behaviors of itself and its measurand/environment using information in its own domain. It also 
provides necessary information for sensor fusion to both the lower HAS-level and the middle 
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Table I 


Measurand Behaviors and related concepts 


SITUATION 

DESCRIPTION 

ASSOCIATED CONCEPTS 

Constant input 

Except for noise, the output remains at a constant level during 
the monitoring time. 

Noise, monitoring time, constant 
level. 

Monotonic 

change 

Except for noise, the output increases or decreases linearly 
with time during the monitoring time 

Noise, linear increase or decrease, 
monitoring time 

Step input 

Except for noise, the output remains constant during the 
monitoring time, then it suddenly jumps to a higher or lower 
level, where it remains during the monitoring time. 

Sudden jump, noise, constant, 
monitoring time. 

Harmonic 

Except for noise, during the monitoring time, the output 
changes at a certain principal frequency. Other secondary 
frequencies may also be present. 

Frequency, principal frequency, 
secondary frequencies, noise. 


process-level. The lower HAS-Level sensor fusion aims at verifying whether the sensor works 
well independent of other sensors and provides reasonable behavior interpretations for the sensor 
and measurand. This is a function of the HAS model alone. The HAS-Bus-Level sensor fusion 
aims at combining higher level qualitative information from all HASs in the Sensor Bus 
associated with the same process. This further improves the interpretations of behaviors of each 
sensor and measurand. A final Process-Bus-Level fusion combines qualitative information 
linking associated processes. Again, this information further improves interpretation of behaviors 
of the sensors and measurands. It is noted that improving interpretations implies learning at the 
sensor and process levels and updating knowledge bases. The overall network should evolve, as 
time passes, into a fine tuned system with increasingly larger and more accurate knowledge bases. 
The top level monitors all processes. It Uses information describing each process at the 
qualitative level to improve the entire network of processes. It may, for example, rearrange the 
schedule for the different processes. 

EFFECTIVE FUSION FOR MONITORING, DIAGNOSIS, AND COMPENSATION 

The Highly Autonomous Sensor (HAS) 

Underlying the generalized distributed and decentralized architecture for multi-sensor fusion is 
the need to understand the nature of the information provided by each sensor. This requires a 
good sensor model detailing each sensor's underlying physical nature and the phenomenological 
nature of its measurements or the qualitative information these measurements provide. A highly 
autonomous sensor (HAS) model developed previously (Figueroa, Yuan, 2000) is a model that 
satisfies these requirements. A formal theory to model a HAS as developed by Figueroa and 
Mahajan (1994a, 1994b, 1995) and expanded by Figueroa and Yuan (2000). This theory is 
implemented in the HAS development environment, and is the essence of an engine for 
qualitative interpretation, reasoning, and decision-making (QIRD) by the sensor. The theory 
permits extraction of qualitative behaviors associated with the sensor, measurand, and 
disturbances. These behaviors, along with expected behaviors denominated “envisionments” and 
other qualitative knowledge that reside in the knowledge bases of the sensor and measurand, are 
used to perform QIRD and evaluate the integrity of the sensor and measurand. The 
envisionments are either entered by the user, or learned by the HAS. 

The most important function of the HAS is the extraction of behaviors. They are defined 
by a succession of “concepts” which are, in turn, defined by properties that maintain constant 
values for a number of samples of the signal being read by the sensor. Table I shows how one 




may determine the concepts that are needed to specify a behavior. For example, given the 
behavior “step change,” a description is generated using concepts that are qualitatively assessed 
by the user. These concepts include constant, noise, sudden jump, and monitoring time. Each of 
these concepts has to be defined in terms of a set of properties and their values at every sample 
time. Sudden jump may be further defined as a combination of two concepts: ramp and fast 
exponential change 

Sensor Fusion 

Processes, in which the network of sensors is involved, must have some mathematical model 
from which qualitative descriptions may be extracted. These descriptions entail qualitative 
relationships among sensors’ readings, which will be used for sensor fusion at the qualitative 
level. A very coarse qualitative description of the process may be sufficient for fusion of 
information from various sensors. For example, if one sensor detects a “step change,” then all 
sensors should experience a “step change” of appropriate intensity. In general, any change in 
measurand behavior should be seen simultaneously by all sensors associated with the same 
process and other related processes, whereas sensor behaviors appear only at individual sensors. 

A theory is being developed to formalize how qualitative relationships among sensor readings 
are represented. This theory is, in first instance, based on sensor and measurand behaviors 
detected by individual HAS’s. It is also inspired by the theory of qualitative simulation 
developed by Kuipers (1986). In order to implement fusion, HAS models must be expanded to 
include information related to other sensors that monitor the same process. For instance, for each 
process a HAS (HAS1) monitors, it must have a module. In each module, there must be a set of 
qualitative relationships among the behaviors of HAS1 and all other HAS’s of the process. The 
first relationship in all modules must indicate that any change in measurand behavior occurs 
simultaneously in all sensors associated with a process. Other relationships may refer to trends, 
such as if the signal from one HAS is changing linearly, another should changing appropriately 
(not necessarily linearly). Noise levels in the various HAS’s should also be related. Thus, 
multiple qualitative relationships may be used to continuously monitor the operation of the 
system and sensors. In this manner, a unusual behavior associated with any HAS can be easily 
detected. In turn, detecting misbehaviors will simplify and speed diagnosis of malfunctioning 
elements. 

Note that fusion is meant to occur at each HAS. Therefore, information need not flow 
through a process, but directly from one HAS to another. This distribution of intelligent 
processing turns each HAS into highly self-sufficient intelligent agents within the context of a set 
a processes and a network of HAS ’s. 

The higher level unit (HLU) receives high level qualitative information from the processes. 

In turn, it sends high level requests and commands aimed at diagnosing a suspected problem or at 
improving the performance of all processes as a unit. For example, given a production line for a 
chemical product, the HLU may receive a message indicating that the product is not “fully cured” 
as it comes out of the last process. The unit may then program longer curing time in the oven 
heating process. In another example, a process may indicate to the HLU that a sensor is suspect 
of drifting behavior due to wear at a high duty-cycle. The HLU may then sound an alarm to have 
the sensor adjusted. 

RESULTS AND DISCUSSION 

An environment to model any sensors as a HAS is fully implemented. The environment has 
been tested with simulated and real data. It has been demonstrated that an individual HAS model 
is able to learn behaviors (measurand and sensor behaviors), store them in its database, and detect 
the learned behaviors when they occur again. Sensor fusion components of the HAS model are 
currently being implemented. A Theory and methodology have been developed, which take 
advantage of the qualitative behaviors extracted by HAS’s distributed in a network of sensors. 


Qualitative reasoning is the key inference mechanism in the data fusion architecture proposed 
in this paper. It has advantages over numeric and statistic methods for sensor data fusion in that: 

(1) it uses a qualitative level of description that permits representation of imprecise knowledge, 

(2) it permits reasoning about inexact systems in that it captures uncertainty by expressing 
multiple behaviors, and (3) it reduces an infinite number of infinitesimally close numeric 
behaviors to a small number of qualitatively distinct behaviors. 

The HAS modeling environment is generic and supports expansion. In fact, it supports 
evolution through learning of behaviors. Aside from the logic rules used to reason with concepts 
and behaviors, other methods could be included such as Fuzzy Logic, Belief Theory, statistical 
methods, neural nets, and evolutionary techniques. One of the main contributions of this work is, 
in fact, that the modeling method is generic and open. Its strength is that it focuses on system 
qualitative behaviors in order to interpret the operation of a sensor and process. And furthermore, 
it exploits these behaviors to perform fusion at highly qualitative levels. In fact, human operators 
monitor and reason about sensors and systems based on qualitative behaviors extracted from the 
signals. This is a very fast and effective method to monitor and diagnose systems. 

From the point of view of computing, the HAS modeling method and the overall structure for 
a network of sensors and processes, conforms to a highly distributed system where the 
information that is transferred among its components is of high level. This greatly reduces the 
bus traffic. 
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